Contextual user profile photo selection

ABSTRACT

Selecting and displaying a profile photo may include determining viewer characteristics of a viewer who is triggering an action to open an image file that shows a photo image. A context associated with viewing of the profile photo and photo image characteristics of a plurality of photos may be determined. Based on the viewer characteristics, the photo image characteristics and the context associated with viewing of the profile photo, a photo from the plurality of photos may be selected. Responsive to determining that the selected photo is to be transformed, a transformed version of the photo image may be generated and displayed.

FIELD

The present application relates generally to computers and computerapplications, and more particularly to selecting and morphing images todisplay based on criteria.

BACKGROUND

User photos can be displayed as an image via many different computerapplications on a computer display device. For instance, a web browserdisplaying a social media web site page can display a user's photo indifferent contexts. Web sites such as a company's directory site or aprofessional site may display a user photo in a professional setting.Individuals have various photos expressing various personas and/ormoods, and the expressions may change depending on who the viewer is,what impression the user would like to convey to the viewer and so on.

BRIEF SUMMARY

A method and system of selecting and displaying a profile photo may beprovided. The method, in one aspect, may include determining viewercharacteristics of a viewer who is triggering an action to open an imagefile comprising a photo image of a user, a group of users, or an object,to view a profile photo of the user, a group of users, and/or an object,the determining performed based on analyzing at least social mediaapplication data. The method may also include determining a contextassociated with viewing of the profile photo. The method may furtherinclude determining photo image characteristics of a plurality of photosstored on at least one storage device. The method may also include,based on the viewer characteristics, the photo image characteristics andthe context associated with viewing of the profile photo, selecting aphoto from the plurality of photos. The method may further includedetermining whether the selected photo is to be transformed. The methodmay also include, responsive to determining that the selected photo isto be transformed, generating a transformed version of the photo imageof the user, the group of users, and/or the object, and displaying thetransformed version of the photo image of the user, the group of users,and/or the object, on a display device associated with the viewer via auser interface. The method may further include, responsive todetermining that the selected photo need not be transformed, displayingthe selected photo on the display device via the user interface.

A system of selecting and displaying a profile photo, in one aspect, mayinclude a hardware processor coupled with a communications network andat least one storage device coupled with the hardware processor. Thehardware processor operable to perform determining viewercharacteristics of a viewer who is triggering an action to open an imagefile comprising a photo image of a user, the group of users, and/or theobject, to view a profile photo of the user, the group of users, and/orthe object, the determining performed based on analyzing at least socialmedia application data. The hardware processor may be further operableto determine a context associated with viewing of the profile photo. Thehardware processor may be further operable to determine photo imagecharacteristics of a plurality of photos stored on the at least onestorage device. The hardware processor may be further operable to, basedon the viewer characteristics, the photo image characteristics and thecontext associated with viewing of the profile photo, select a photofrom the plurality of photos. The hardware processor may be furtheroperable to determine whether the selected photo is to be transformed.The hardware processor may be further operable to, responsive todetermining that the selected photo is to be transformed, generate atransformed version of the photo image of the user, the group of users,and/or the object, and trigger a displaying of the transformed versionof the photo image of the user, the group of users, and/or the object,on a display device associated with the viewer via a user interface. Thehardware processor may be further operable to, responsive to determiningthat the selected photo need not be transformed, trigger a displaying ofthe selected photo on the display device via the user interface.

A computer readable storage medium storing a program of instructionsexecutable by a machine to perform one or more methods described hereinalso may be provided.

Further features as well as the structure and operation of variousembodiments are described in detail below with reference to theaccompanying drawings. In the drawings, like reference numbers indicateidentical or functionally similar elements.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating components of a system ofselecting and displaying photos in one embodiment.

FIG. 2 is a diagram illustrating a user profile store in one embodiment.

FIG. 3 is a flow diagram illustrating a method of automaticallyselecting and displaying a photo in one embodiment.

FIG. 4 illustrates a schematic of an example computer or processingsystem that may implement a system in one embodiment of the presentdisclosure.

DETAILED DESCRIPTION

Method, system and techniques are provided, which display a profilephotograph (e.g., on a World Wide Web's web page) and automaticallyselect one or more photographs to display based on an analysis of aviewer of the photographs and characteristics of available user photosand/or analysis of various context (e.g., user context and geo-locationcontext).

A person usually has photographs (also referred to as photos) of thatperson which present various different personas and context (e.g.,expressions, sympathy, condolences). Different photos may be presentedto different viewers depending on various criteria. For example, onsocial media's web pages, photographs are often used. Photos or evenplayful drawings of users are used on various computer displays such asweb pages, and even at web sites such as resume sites and businesssites, for example, for purposes.

In some embodiments, the web page with the photo may be a social mediaweb page or a company's directly web page, a job site's web page, and/orothers. The viewer analysis performed by the system and/or method of thepresent disclosure in some embodiments may include an estimation ofviewer demographic; geographic location (e.g., culturally sensitiveplace) and context of the viewer (e.g., expression, sympathy,condolences); service provider of the viewer; interests of viewer, roleof the viewer (e.g., friend, a manager at a company, a job recruiter ora bot recruiter); the number of page views for individual pages, whichindicates approximately how many people are visiting the site displayingthe photo and which pages on the site are the most popular; duration oftime people spend on specific pages, which indicates information such aswhether a home page is keeping people's interests, what sites peoplewere visiting before they visited the site, which help in understandingwhether the web traffic is coming from links on other web pages, fromsearches, and/or others; when (e.g., the time) people visit the site andhow long they stay; what countries the site visitors are from, forexample, to determine types of dress code (e.g., liberalness,conservativeness), what browsers and operating systems the visitors areusing; and/or others.

Characteristics of photos may involve any one or more of: facialexpression and mood, cropping, magnification, size, clothing, colorthemes, hair style, lighting, position, posture, viewpoint, backgroundimage, attractiveness estimation based on crowdsourcing (e.g., voting),the person photographed alone or with others (e.g., spouse or another),photorealism (e.g., a real photo, an avatar, a drawing or sketch),and/or animation.

Additional context may include time and place a viewer is opening aprofile and photo, e.g., in a public place such as culturally sensitiveplace, a place where hijab is required, and/or others, time of day,holiday, and/or day of week.

Information on the viewer (e.g., of the photo on a Web page, socialmedia app) may be performed in an opt-in fashion and/or may be gleanedfrom a social media profile such as those shown on a social mediaapplication's web page or wall posting. The photos may be pre-loaded bya user. For instance, a user may provide the system with a number ofphotos (e.g., 5 photos) from which to choose.

The method and/or system for dynamic user profile photo selection insome embodiments may use user-defined geotagged rules in the userpreference settings. Examples of rules include, but are not limited to:

Rule 1: If a location L is deemed to be crowded, use photo 3 of 7.

Rule 2: If a location L is deemed to be privacy sensitive, generateavatar to replace my real photo.

Rule 3: If a viewer context is determined/detected to be “Happy”, selectphotos that are tagged with “happy.”

In some embodiments, the rules can be learned based on historicalinformation. For instance, a rule engine may use variousgeolocation-dependent analytic services. Such services may include acrowd density analytic that may use nearby beacons, WiFi (wirelessnetworking), connected devices, and/or others; social-networkinganalysis that learns about privacy associated risks to the user to agiven location; deep-learning services to generate avatar bytransforming the real photo of the user for privacy protection.

In some embodiments, the rules may be learned based on historicallocation-dependent context information, users, viewing or browsinghistory, social medial history. New rules, for example, which arelearned automatically or autonomously, may be presented to the user on agraphical user interface (GUI) and/or added to the preference settingsonce approved by the user.

In some embodiments, the system and/or method may change or morph theprofile picture based on a particular geotagging, location context,e.g., to express or show sympathy or condolences. Such context can belearned from analyzing social medial and crowdsourced data as well astriangulated with geolocation information.

In some embodiments, the profile photo may be upgraded or modifiedaccording to the viewer's feedback on the profile photo to achieve asuitable response from the viewer. Feedback, for example, may beattained from social media websites, for example, in forms of “likes”(an indication of approval for the post) and/or other indications,comments, sharing the photo with other people along with theirdescription. Feedback, for example, may also be received from detectingthe viewer's eye movements by eye tracking, gaze pattern analysis, theamount of time the viewer has a photo displayed, biometric feedback fromfacial expressions, and/or others. Automated feedback of this kind maybe gained in a privacy preserving manner such as when a user (e.g.,viewer) opts-in, for instance, with permission or authorization from theuser.

The system and/or method of the present disclosure in some embodimentsmay automatically interpret the viewer's response and modify an existingimage or select a new image (e.g., fetch from cloud or user's linkeddevices or another). An image may also be modified based on thepriorities and preferences set by the user, so as to display a newprofile photo to attain a set objective, e.g., for the next time thesame viewer visits the user's web page or the like that displays theuser's photo.

For example, a user in their profile may store multiple photos showingthe user in different settings. For instance, a first photo of the userwith another user, a second photo of the user which shows the userwearing university garb, a third photo of the user in a businesssetting, a fourth photo of the user in more casual setting, a fifthphoto of the user which displays the user is speaking with a microphoneat a podium in front of an audience. There may be different photos withdifferent facial expressions. There may be different photos withdifferent posture and clothing. The system and/or method of the presentdisclosure in some embodiments may select one of the photos stored inthe user's profile for display based on an analysis of context and theviewer viewing the photo.

In some embodiments, the system and/or method of the present disclosuremay select automatically different qualities of images based on context,for example, in case of an intermittent network connection. In someembodiments, an image quality subsystem may compute quality scores forimages that represent a measure of visual quality of the images. Qualitymay refer to resolution, sharpness, and other aspects.

In some embodiments, the system may trigger a nearby user device for anew photo by transmitting information such as context, a rule, and/oranother criterion. A user device may be recognized as being nearby, forexample, if the system is able to communicate with the user device, forexample, via short range communication techniques such as Bluetooth, orfor example, via a local area network (LAN) or wide area network (WAN),or another. In another aspect, the user device that is triggered mayneed not be within a short range of communications, for example, so longas the system is able to communicate with the user device, for example,via a wireless network (WiFi) or another network. Examples of a userdevice may include, but are not limited to, a smartphone, a mobilephone, a tablet, and/or others. Responsive to receiving the trigger, theuser device may select a suitable photo from a photo gallery (e.g.,photos stored on the user's device memory, or photos stored elsewhereaccessible by the user device, e.g., via WiFi), establish communications(e.g., Bluetooth) and then securely transmit the selected photo tochange the user profile.

In some embodiments, the automated system and/or method of user profilephoto selection may be triggered based on a user command such as a voicecommand. For instance, a viewer may voice a command that indicates theviewer's current need-based profile picture selection. As an example, aviewer may have a plan to attend a party and would like to see photoswith a certain hairstyle, for example, curly hairstyle that a friend orcoworker used before. The system and/or method of the present disclosuremay then dynamically select photos that match curly hairstyles andpresent to the viewer.

The system and/or method of the present disclosure may also optionallyimplement a psycholinguistics analytic module to generatepsycholinguistics personality profile of a viewer to find and/or suggestthe types of profile picture. The psycholinguistics personality profileof the viewer may be built also in conjunction with an analysis of the“owner” of the profile page. The analytics module may gather recent pastsocial network contributions of viewer (for example, including what heposts and/or replies) and then based on psycholinguistics analysis, thesystem and/or method of the present disclosure may identify recentpersonality profile of the user. An example of a determined personalityprofile of a viewer may be that the viewer is “adventurous.”

Tools such as the IBM® Watson Personality Insights service, which usesIBM® Bluemix services may be employed to determine a personalityprofile. For instance, the IBM® Watson Personality Insights service useslinguistic analytics to infer the personality traits, intrinsic needs,and values of individuals from communications that the user makesavailable via mediums such as email, text messages, blog messages, forumposts, and/or others. The tool includes visualizing the analysis withtypes of information displayed in color-coded sections, and additionallywith percentage indications for each characteristic in a personalityprofile. By way of another implementation, visual analytics techniques(e.g., deep-neural network) can be used to infer the personality traitsof the user by analyzing facial expression, eye movement, gaze pattern,and/or others.

In some embodiments, traits or characteristics above a thresholdpercentage may be used in obtaining a profile picture. In someembodiments, color and clothing style preferences may be considered, forexample, including the color and clothing style preferences of a useraccessing a Web site. Varying degree of photorealism may be utilized,for example, from an actual photo of a person to modified images.

A user profile photo may apply to a team, for example, more than oneperson in a photo. The system and/or method in such scenarios may searchfor a photo in which all or part of a group is shown with the desiredtraits or profile. In some embodiments, the system may use customtrained machine learning models (e.g., trained deep-learning modelsusing historical data) to compose a group photo by combiningindividual's photos, wherein the individual photos may be retrieved fromuser's contact directory, their websites, social medial pages, and/orothers.

The system and/or method of the present disclosure may be used by anartificial intelligent (AI) agent. For example, an automated robot witha presence on the Web with a profile photo may desire to change theprofile photo based on various analyses, for example, analyzing viewerof the photo, characteristics of available user photos, and geo-locationcontext.

FIG. 1 is a block diagram illustrating components of a system ofselecting and displaying photos in one embodiment. An analytics module102, photo manipulation module 104, photo and profile selector 106 anddisplay manager 108 components run or execute on one or more hardwareprocessors, for example, coupled with one or more memory devices. Thedisplay manager 108 that manages the photo manager 136 and graphicaluser interfaces (GUI) 134, which may display photos, for example, on aweb site page, and/or a mobile device, for example, on a display screenof a device.

The analytics module 102 may include functionalities that performcontext analytics 110, social media analytics 112, personality analytics114 and psycholinguistics analytics 116.

Context analyzer 110 may determine context about a user, a location, andphotos. For example, cultural norm of a user and a viewer at a givenlocation L, interest of a viewer, environment context of the user and aviewer and other context information. The context analyzer 110 maydetermine the time and place a viewer is triggering a displaying oropening of a user's photo, e.g., a web page or an application page thatdisplays the user's photo. The context analyzer 110 may use historicalgeolocation data and crowd-sourced data to understand and characterizethe cultural norm at a given location L. The context analyzer may usethe personality analyzer 114, social media analyzer 112, and/orpsycholinguistics analyzer to further understand interests of a viewerat a given time and location, context of a user (e.g., expression,sympathy, condolences) and so on.

Social media analyzer 112 may extract relevant information from socialmedia applications to dynamically select and configure photos for aviewer. For example, viewer and user profiles, viewer and/or useractivities on social media web pages, for example, web pages the viewerand/or users view, the comments the viewer and/or user posts, and/orother activities on social media web sites. Various existing algorithmssuch as predictive analytics, statistical and machine learningalgorithms maybe used to train these algorithms using extracted socialmedia data and recognize the patterns of a language or an image that apost is about cultural norms at a given location and time.

Personality analyzer 114 determines viewer and/or user personality, forexample, based on information from the context analytics, social mediaanalytics, psycholinguistics analytics, analysis of historical datarelated to a viewer engagement and interaction over a period of time,for a facial analysis, eye movement, gaze pattern analysis, and/orothers. The method of personality analysis may receive and analyzeviewer feedback by executing a custom trained machine learning modelsuch as a neural network model.

Psycholinguistics analyzer 116 determines psycholinguistics personalityprofile of a viewer, for example, as described above, for example,employing tools such as the IBM® Watson Personality Insights servicethat infers personality traits. Psycholinguistics analyzer 116 may inputinformation extracted by the social media analytics 112 to such a toolto generate personality traits.

Photo manipulation module 104 may manipulate photos and may includefunctionalities such as a rule engine 118, morphing engine 120, profilematcher 122 and photo receiver 124. The rule engine 118 may determineone or more rules which encode the specifications of what profile/photosshould be selected, whether the profile and/or a photo should bemodified, or an avatar or a drawing should be generated to display inplace of displaying the photo. One or more rules may be fetched from aset of predetermined rules (e.g., user specified rules) and/or generatedby applying one or more rule generation algorithms that may use one ormore machine learning algorithms to learn from historical data. Forexample, a rule-based machine learning method (e.g., association rulelearning) maybe employed to discover interesting relations betweenprevious rules in rule database, features in geolocation/demographicdatabase, features in a viewer profile, features in a user profile photostore.

The photo receiver 124 may receive a photograph, e.g., an image file,from a photo or profile component 106 or from a nearby user device. Themorphing engine 120 may modify the photograph, e.g., change the pixelvalues of the image file, change the color or brightness of the image,transform (e.g., blur or hide) part of the image that may containsensitive or inappropriate information, based on a rule generated by therule engine 118 or inputs from at least on analytics module 102.

The profile matcher 122 receives profile and/or photo from the photoreceiver 124 and is responsible for matching the profile and/or photowith respect to the requirements or factors received from the profileand/or photo configuration engine 126. The profile matcher 122 mayinteract with the analytics module 102 to further refine and optimizephotos which initially have passed the matching requirements or factors.In one embodiment, the profile matcher may trigger the morphing engine120 depending on the analysis of the matching results (e.g., if thequality of matched photos is deemed to be below a specified threshold,the morphing engine maybe triggered).

The profile selector component 106 may select or retrieve a photo from auser profile store or database, for example, shown in FIGS. 2 and 212,for example, based on the rule engine's rule, and send the selectedphoto to the photo manipulation module 104. The profile selectorcomponent 106 may also receive a modified photo (if any) and display themodified photo or the photo via a display manager 108 to a displaydevice or like interface such as a graphical user interface (GUI) 134,for example, of a web page, a mobile device or the like.

The profile selector component 106 may include a profile configuringengine 126, an S&P controller 128, a view monitoring component 130, andthe display controller 132. The profile configuring engine 126configures a profile and/or photo by interacting with the photomanipulation module 104, context analyzer 110, and/or based on analysisof data received from the view monitoring 130. The process ofconfiguring a profile may include receiving requirements or factors togenerate a profile for the first time, interpreting a viewer feedback tomodify a profile photo(s), communicating with the photo manipulationmodule 104 to fetch a profile or photo.

The S&P controller 128 controls the Security and Privacy (S&P) aspectsof a user photo (and the user) and/or a viewer by interacting with theanalytics module 102 for inputs such as determined context, geolocationinformation, demographics characteristics to determine the security andprivacy requirements, which are communicated with the display controller132.

The view monitoring component 130 may collect stream of data about aviewer such as interaction and engagement (e.g., clickstream, timespent, and/or others) of a viewer with each displayed photo through anon-intrusive instrumentation and monitoring methods, e.g., Google®Analytics tools. In one aspect, such monitoring is performed withpermission of the viewer or another being monitored.

The display controller 132 manages models and logics needed for thedisplay manager 108 at runtime. The display manager may includepreconfigured models and logics (e.g., S&P requirements).

The display manager 136 controls the photo manager 136 using the modelsand logics supplied by the display controller 132. For example, thedisplay manager 136 may determine, at runtime, how the profile and/orphoto(s) should be displayed on the Graphical User Interface (GUI) 134according to at least one runtime condition such as device size andnetwork connection. The components of shown in FIG. 2 may be implementedbased on client-server architecture, for example, in which the clientside instruments, collects and transmits data to the backend system aswell as displays the photo on GUI. The backend system may be implementedon a Cloud computing environment and/or a proprietary system, or anothercomputing system. The backend system may execute the various embodimentsdescribed herein. In some embodiment, one or more processingcapabilities may reside on the client side, e.g., based on theavailability of computation resources such as central processing unit,storage, and/or other resources. In one aspect, the GUI 134 may resideon client side.

FIG. 2 is a diagram illustrating a user profile store, for example,repository of photos (e.g., from a user's social media site, photogallery store, e.g., on a mobile phone, and/or another storage) in oneembodiment. A user profile photo may be retrieved from various datastores and web sites, for example, by performing a people search on theInternet or the World Wide Web as shown at 202, searching the user's ownsite 204, such as the owner's own web site or web page, for example, ona social media application or platform, other web sites 206 and/orcustom profiles 208. Each profile photo and data retrieved or receivedfrom the various data sources may be analyzed using Feature Extraction &Labeling Module 210 so as to extract one or more characteristics of aprofile and/or photo and one or more features of a profile and/or photo.One or more characteristics of a profile and/or photo may include dateand location of the photo taken, description of the photo, context ofthe photo such as taken during an event as such a party. One or morefeatures of a profile and/or photo may include one or more derivedvalues of the photo (e.g., the sentiment of the user, cognitive state ofthe user such as happy, stress, confused, emotional level, and/orothers) of the photo. One or more derived values of the photo may beextracted from the profile and/or photo by applying one or more of aplurality of custom machine learning algorithms or models such as edgedetection, histogram computation, deep learning algorithms, sparsecoding methods, and/or others. For example, the Feature Extraction &Labeling Module 210 may use a statistical association rule miningalgorithm to select the most relevant profile image features. In furtherembodiment, the extracted characteristics, features and other metadatainformation in relation to the photo may be securely (e.g., usingencryption technique) stored in a user profile store 212, along with thephotos, which are also tagged to facilitate dynamic searching andindexing. The user profile store 212 may be hosted on a Cloud ordedicated (on-premise) storage device or environment. The user profilestore 212 may include data stored as a database system, and stored on astorage device.

The system and/or method of the present disclosure may be used by one ormore of cross-vendor social media website or services or artificialintelligent (AI) agents.

FIG. 3 is a flow diagram illustrating a method of automaticallyselecting and displaying a photo in one embodiment. The method may beexecuted by one or more hardware processors, for example,communicatively coupled to a communications network, and operativelycoupled to at least on storage device. At 302, viewer characteristics ofa viewer who is trying to view a profile photo of a user are determined.For instance, the viewer may be triggering an action to display or openan image file that includes a photo image of the user, for example,requesting of opening the file by visiting a web site or web page thatdisplays a photo, or another action that would display the image file.The viewer characteristics may be determined by analyzing data, forexample, social media application data and/or other data. A user mayinclude a group of users, for example, a team. In another aspect, a usermay include a user's organization, software agent. In those examples, aphoto of a user may include a photo of a team at work, family members,company, a store, and/or a graphic or icon that represents a bot. Asanother example, the photo that is being requested for display for aviewer to view may include a building or another object.

Examples of viewer characteristics may include an estimation of theviewer demographic, a service provider of the user, interests of theuser, geographic location of the user, the number of page views forindividual pages of the user, duration of time people spend on specificpages of the user, the web sites people were visiting before theyvisited the site that displays the profile photo, the time peoplevisited the site that displays the profile photo and how long they stayon the site, from what countries the visitors to the site are, forexample, to determine liberalness of dress code, what computing device(e.g., a smartphone, tablet, standard computer, and/or another device),browsers and operating systems the visitors to the web site are using.The method of photo selection process may receive the requirements orfactors for selecting a photo(s) and apply the selection algorithm toretrieve from user profile store 212 the best matching profile and/orphotos according to the received requirements or factors. If a matchingprofile and/or photo is not found or the confidence level of theselected photo is below a defined threshold, the selection algorithm maytrigger one or more other techniques such as triggering a nearby orremote user device for a new photo by sending the selection factors,and/or triggering a morphing engine, and/or another action.

At 304, a context associated with viewing of the profile photo may bedetermined. Such context may be determined by analyzing social mediadata and/or from crowdsourced data. The context may include culturalnorm context and environmental context, for example, time and place aviewer is opening the profile photo, for example, in a public place,culturally sensitive place, time of day, day of week, holiday, and/orothers.

At 306, photo image characteristics of a plurality of photos stored onat least one storage device may be determined. For instance, imageanalysis may be performed on the plurality of photos to identifydifferent characteristics of the photos. The photo characteristics mayinclude various attributes of the photo such as: facial expression,microexpression (e.g., mood analysis), cropping/magnification, size,clothing, color themes, fashion styles of individuals appearing in thephotos, lighting, position, posture, viewpoint, background image,attractiveness estimation based on crowdsourcing (e.g., voting), numberof individuals appearing in the photos, the degree or extent ofphotorealism (e.g., a real photo, an avatar, a drawing), an animation.

The plurality of photos from which to choose may be received via acommunications network from one or more of a mobile device, a serverdevice, and a web server device. In another aspect, the plurality ofphotos may have been pre-loaded by the user, and made accessible by thehardware processor executing the method.

At 308, based on the viewer characteristics, the photo imagecharacteristics, the photo image features and the context associatedwith viewing of the profile photo, a photo is selected from theplurality of photos stored in the user profile store 212. In someembodiments, a photo may be selected further based on one or more rulesspecifying one or more conditions in selecting the photo.

At 310, it is determined whether the selected photo is to betransformed. For example, based on the context such as requirement forprivacy, it may be determined that the selected photo should betransformed before being displayed. For instance, it may be determinedthat the photo should be transformed the viewer's previous responses tophotos the viewer has viewed, and/or based on analysis of the viewerprofile, and/or others. The user may also have specified to not displaythe selected photo as is. In another aspect, the method may learn, e.g.,using a machine learning algorithm and training at least one customtrained machine learning model, the context, user feedback, risk, andother factors from historical data and other plurality of data sources(e.g., crowdsourcing) such that at least one transformation algorithmmay be executed to dynamically adapt avatar that complies withrequirements, for example, with privacy and security requirements andviewer concern level.

At 312, responsive to determining that the selected photo is to betransformed, a transformed version of the photo image of the user isgenerated and the transformed version of the photo image of the user isdisplayed on a display device associated with the viewer via a userinterface. For example, an image of an avatar corresponding to the usermay be generated to be displayed in lieu of the actual user's photo. Asanother example, an image of the user with a selected degree ofphotorealism may be generated. The degree of photorealism that isselected may be based on the accuracy level of the at least one customtrained machine learning model for photo morphing, a threshold value setfor the number of iteration to be done, user feedback, and/or otherfactors. Techniques such as deep-learning services may be employed togenerate an avatar image by transforming the real photo of the user, forexample, for privacy protection. In another aspect, the user's photo maybe changed or morphed based on the context, e.g., to express sympathy orcondolences.

At 314, responsive to determining that the selected photo need not betransformed, the selected photo is displayed on the display device viathe user interface.

A profile photo of a user may be displayed, for example, at social mediaweb pages such as the web pages of existing social media applications, acompany phone book, a resume page, and/or others.

In some embodiments, one or more rules may be automatically determinedfor specifying one or more conditions in selecting the photo, forexample, what photos should be selected, whether the profile and/or aphoto should be modified, or an avatar or a drawing should be generatedto display in place of displaying the photo. One or more rules may befetched from a set of predetermined rules (e.g., user specified rules)and/or generated by applying one or more rule generation algorithms thatmay use machine learning algorithms (e.g., a genetic algorithm,association rule learning) to automatically learn from historical dataavailable from one or more data sources. For example, a rule-basedmachine learning method maybe employed to discover interesting relations(e.g., photo matching) between historical rules in a rule database,features in geolocation/demographic database, features in a viewerprofile, and/or features in a user profile photo store. One or morerules may be based on historical data, one or more of the social mediaapplication data, determined context, and/or other data. Rules may belearned from historical information, crowd density informationdetermined using crowd density analytics performed using nearby beacons,WiFi, and/or other connected devices; and/or information fromsocial-networking analysis to learn about privacy associated risk to theuser to a given location.

In another aspect, a photo may be selected further based on user-definedgeotagged rules that can be configured on user preference settings suchas firewall rules. An example of a rule may specify, “If a location L isdeemed to be crowded, use photo 3 of 7”; another example rule mayspecify, “If a location L is deemed to be privacy sensitive, generate anavatar to replace my real photo.”

The method may also include triggering a nearby or remote user devicesuch as a mobile phone, a tablet, a laptop, or another device, for newphoto by transmitting information such as the context and/or a rule. Themethod of triggering a nearby or remote user devices may be determinedbased on at least one requirement received from a photo selectionalgorithm, which may reside on the client side or server side of thedisclosed system. An application running on the remote user device mayreceive the information and based on the information select a suitablephoto from a photo gallery, establish a communication (e.g., Bluetooth)if not already established, and then securely transmit the selectedphoto to change the user profile.

The method may also include detecting or monitoring, aggregating andanalyzing viewer's feedback on the profile photo. In one aspect,feedback may be attained from social media websites in the form of“Likes”, comments entered by the viewer, sharing the photo with otherpeople along with their description, a voice command, implicit feedback,and/or others. Based on the feedback, the profile photo may bedynamically upgraded, for example, to a new photo, morphed or replacedinto a different image. The viewer's feedback, for example, of the photoor a modified photo, may be used to establish a confidence level of thenew photo. For instance, in some embodiments, the confidence level isestablished based on the viewer's feedback. Positive and negativecomments and reactions to different photos (including the one selectedto be displayed) may be fed back as inputs to a neural networks machinelearning algorithm in order to learn the optimal photo selection. Forinstance, initially, the photo displayed may be considered to be notappropriate (relatively) based on only the input features like viewer'sprofile, social media portal information and other factors. Thereal-time feedback implies analysis of real-time comments or reactions(such as “likes”) to the photo which can modify the output at any giventime. Hence, this leads to selecting an alternate photo or filtering orblurring (hiding) the contents of the photo based on that activefeedback which is inherently modifying the confidence level (or rigidityfactor as alternate term to be used) of the machine learning algorithm.The information on the viewer is obtained with the permission of theviewer, for instance, in an opt-in fashion.

The viewer's response to a modified profile photo may be interpretedautomatically by the system in order to modify an existing photo orselect a new photo. The method of interpreting a viewer feedback mayreceive a viewer response to a modified profile photo. A viewer responsemay be an explicit response (e.g., a text or audio command or feedback),implicit response, or a combination of both. Examples of explicitresponse include a viewer confusion, dislike or satisfaction level,which may be dynamically detected from viewer monitoring patterns,analysis of facial expression, eye movement, voice tone, and/or others.In another aspect, an existing photo may be modified based on predefinedpriorities and preferences set by the user. In some embodiments, basedon the result(s) of interpretation of a viewer response, the method mayinclude selecting a new photo, applying at least one custom trainedmachine learning model to modify an existing photo, for example,morphing an existing photo.

FIG. 4 illustrates a schematic of an example computer or processingsystem that may implement a system in one embodiment of the presentdisclosure. The computer system is only one example of a suitableprocessing system and is not intended to suggest any limitation as tothe scope of use or functionality of embodiments of the methodologydescribed herein. The processing system shown may be operational withnumerous other general purpose or special purpose computing systemenvironments or configurations. Examples of well-known computingsystems, environments, and/or configurations that may be suitable foruse with the processing system shown in FIG. 4 may include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, handheld or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

The computer system may be described in the general context of computersystem executable instructions, such as program modules, being executedby a computer system. Generally, program modules may include routines,programs, objects, components, logic, data structures, and so on thatperform particular tasks or implement particular abstract data types.The computer system may be practiced in distributed cloud computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed cloudcomputing environment, program modules may be located in both local andremote computer system storage media including memory storage devices.

The components of computer system may include, but are not limited to,one or more processors or processing units 12, a system memory 16, and abus 14 that couples various system components including system memory 16to processor 12. The processor 12 may include a module 30 that performsthe methods described herein. The module 30 may be programmed into theintegrated circuits of the processor 12, or loaded from memory 16,storage device 18, or network 24 or combinations thereof.

Bus 14 may represent one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnects (PCI) bus.

Computer system may include a variety of computer system readable media.Such media may be any available media that is accessible by computersystem, and it may include both volatile and non-volatile media,removable and non-removable media.

System memory 16 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) and/or cachememory or others. Computer system may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 18 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(e.g., a “hard drive”). Although not shown, a magnetic disk drive forreading from and writing to a removable, non-volatile magnetic disk(e.g., a “floppy disk”), and an optical disk drive for reading from orwriting to a removable, non-volatile optical disk such as a CD-ROM,DVD-ROM or other optical media can be provided. In such instances, eachcan be connected to bus 14 by one or more data media interfaces.

Computer system may also communicate with one or more external devices26 such as a keyboard, a pointing device, a display 28, etc.; one ormore devices that enable a user to interact with computer system; and/orany devices (e.g., network card, modem, etc.) that enable computersystem to communicate with one or more other computing devices. Suchcommunication can occur via Input/Output (I/O) interfaces 20.

Still yet, computer system can communicate with one or more networks 24such as a local area network (LAN), a general wide area network (WAN),and/or a public network (e.g., the Internet) via network adapter 22. Asdepicted, network adapter 22 communicates with the other components ofcomputer system via bus 14. It should be understood that although notshown, other hardware and/or software components could be used inconjunction with computer system. Examples include, but are not limitedto: microcode, device drivers, redundant processing units, external diskdrive arrays, RAID systems, tape drives, and data archival storagesystems, etc.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements, if any, in the claims below areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

We claim:
 1. A method of selecting and displaying a profile photo, themethod performed by at least one hardware processor, the methodcomprising: determining viewer characteristics of a viewer who istriggering an action to open an image file comprising a photo image of auser to view a profile photo of the user, the determining performedbased on analyzing at least social media application data, the viewercharacteristics including at least a geolocation of the viewertriggering the action to open the image file; determining a contextassociated with viewing of the profile photo; determining photo imagecharacteristics of a plurality of photos stored on at least one storagedevice; based on the viewer characteristics, the photo imagecharacteristics and the context associated with viewing of the profilephoto, selecting a photo from the plurality of photos; determiningwhether the selected photo is to be transformed; responsive todetermining that the selected photo is to be transformed, generating atransformed version of the photo image of the user and displaying thetransformed version of the photo image of the user on a display deviceassociated with the viewer via a user interface; and responsive todetermining that the selected photo need not be transformed, causingdisplaying of the selected photo on the display device via the userinterface.
 2. The method of claim 1, further comprising automaticallygenerating one or more rules specifying one or more conditions inselecting the photo based on one or more of the social media applicationdata and the context.
 3. The method of claim 1, wherein the selecting aphoto from the plurality of photos is performed further based on one ormore rules specifying one or more conditions in selecting the photo. 4.The method of claim 1, wherein the generating the transformed version ofthe photo image comprises generating an image of an avatar correspondingto the user.
 5. The method of claim 1, wherein the generating thetransformed version of the photo image comprises generating an imagewith a degree of photorealism corresponding to the user.
 6. The methodof claim 1, wherein the plurality of photos stored on at least onestorage device is received via a communications network from one or moreof a mobile device, a server device, and a web server device.
 7. Themethod of claim 1, wherein the plurality of photos is pre-loaded by theuser.
 8. A computer program product comprising a computer readablestorage medium having program instructions embodied therewith, theprogram instructions executable by a device to cause the device to:determine viewer characteristics of a viewer who is triggering an actionto open an image file comprising a photo image of a user to view aprofile photo of the user, the determining performed based on analyzingat least social media application data, the viewer characteristicsincluding at least a geolocation of the viewer triggering the action toopen the image file; determine a context associated with viewing of theprofile photo; determine photo image characteristics of a plurality ofphotos stored on at least one storage device; based on the viewercharacteristics, the photo image characteristics and the contextassociated with viewing of the profile photo, select a photo from theplurality of photos; determine whether the selected photo is to betransformed; responsive to determining that the selected photo is to betransformed, generate a transformed version of the photo image of theuser and displaying the transformed version of the photo image of theuser on a display device associated with the viewer via a userinterface; and responsive to determining that the selected photo neednot be transformed, cause to display the selected photo on the displaydevice via the user interface.
 9. The computer program product of claim8, where in the device is further caused to automatically generate oneor more rules specifying one or more conditions in selecting the photobased on one or more of the social media application data and thecontext.
 10. The computer program product of claim 8, wherein the deviceis caused to select a photo from the plurality of photos further basedon one or more rules specifying one or more conditions in selecting thephoto.
 11. The computer program product of claim 8, wherein the deviceis caused to generate the transformed version of the photo image bygenerating an image of an avatar corresponding to the user.
 12. Thecomputer program product of claim 8, wherein the device is caused togenerate the transformed version of the photo image by generating animage with a degree of photorealism corresponding to the user.
 13. Thecomputer program product of claim 8, wherein the plurality of photosstored on at least one storage device is received via a communicationsnetwork from one or more of a mobile device, a server device, and a webserver device.
 14. The computer program product of claim 8, wherein theplurality of photos is pre-loaded by the user.
 15. A system of selectingand displaying a profile photo, comprising: a hardware processor coupledwith a communications network; and at least one storage device coupledwith the hardware processor; the hardware processor operable to perform:determining viewer characteristics of a viewer who is triggering anaction to open an image file comprising a photo image of a user to viewa profile photo of the user, the determining performed based onanalyzing at least social media application data, the viewercharacteristics including at least a geolocation of the viewertriggering the action to open the image file; determining a contextassociated with viewing of the profile photo; determining photo imagecharacteristics of a plurality of photos stored on the at least onestorage device; based on the viewer characteristics, the photo imagecharacteristics and the context associated with viewing of the profilephoto, selecting a photo from the plurality of photos; determiningwhether the selected photo is to be transformed; responsive todetermining that the selected photo is to be transformed, generating atransformed version of the photo image of the user and triggering adisplaying of the transformed version of the photo image of the user ona display device associated with the viewer via a user interface; andresponsive to determining that the selected photo need not betransformed, triggering a displaying of the selected photo on thedisplay device via the user interface.
 16. The system of claim 15, wherein the hardware processor further automatically generates one or morerules specifying one or more conditions in selecting the photo based onone or more of the social media application data and the context. 17.The system of claim 15, wherein the hardware processor selects a photofrom the plurality of photos further based on one or more rulesspecifying one or more conditions in selecting the photo.
 18. The systemof claim 15, wherein the hardware processor generating the transformedversion of the photo image comprises the hardware processor generatingan image of an avatar corresponding to the user.
 19. The system of claim15, wherein the hardware processor generating the transformed version ofthe photo image comprises the hardware processor generating an imagewith a degree of photorealism corresponding to the user.
 20. The systemof claim 15, wherein the plurality of photos stored on at least onestorage device is received via the communications network from one ormore of a mobile device, a server device, and a web server device.